Large language models for biomedicine: foundations, opportunities, challenges, and best practices.

OBJECTIVES: Generative large language models (LLMs) are a subset of transformers-based neural network architecture models. LLMs have successfully leveraged a combination of an increased number of parameters, improvements in computational efficiency, and large pre-training datasets to perform a wide spectrum of natural language processing (NLP) tasks. Using a few examples (few-shot) or no examples (zero-shot) for prompt-tuning has enabled LLMs to achieve state-of-the-art performance in a broad range of NLP applications. This article by the American Medical Informatics Association (AMIA) NLP Working Group characterizes the opportunities, challenges, and best practices for our community to leverage and advance the integration of LLMs in downstream NLP applications effectively. This can be accomplished through a variety of approaches, including augmented prompting, instruction prompt tuning, and reinforcement learning from human feedback (RLHF). TARGET AUDIENCE: Our focus is on making LLMs accessible to the broader biomedical informatics community, including clinicians and researchers who may be unfamiliar with NLP. Additionally, NLP practitioners may gain insight from the described best practices. SCOPE: We focus on 3 broad categories of NLP tasks, namely natural language understanding, natural language inferencing, and natural language generation. We review the emerging trends in prompt tuning, instruction fine-tuning, and evaluation metrics used for LLMs while drawing attention to several issues that impact biomedical NLP applications, including falsehoods in generated text (confabulation/hallucinations), toxicity, and dataset contamination leading to overfitting. We also review potential approaches to address some of these current challenges in LLMs, such as chain of thought prompting, and the phenomena of emergent capabilities observed in LLMs that can be leveraged to address complex NLP challenge in biomedical applications.

Identifying Unmet Needs in Major Depressive Disorder Using a Computer-Assisted Alternative to Conventional Thematic Analysis: Qualitative Interview Study With Psychiatrists.

BACKGROUND: The development of digital health tools that are clinically relevant requires a deep understanding of the unmet needs of stakeholders, such as clinicians and patients. One way to reveal unforeseen stakeholder needs is through qualitative research, including stakeholder interviews. However, conventional qualitative data analytical approaches are time-consuming and resource-intensive, rendering them untenable in many industry settings where digital tools are conceived of and developed. Thus, a more time-efficient process for identifying clinically relevant target needs for digital tool development is needed. OBJECTIVE: The objective of this study was to address the need for an accessible, simple, and time-efficient alternative to conventional thematic analysis of qualitative research data through text analysis of semistructured interview transcripts. In addition, we sought to identify important themes across expert psychiatrist advisor interview transcripts to efficiently reveal areas for the development of digital tools that target unmet clinical needs. METHODS: We conducted 10 (1-hour-long) semistructured interviews with US-based psychiatrists treating major depressive disorder. The interviews were conducted using an interview guide that comprised open-ended questions predesigned to (1) understand the clinicians' experience of the care management process and (2) understand the clinicians' perceptions of the patients' experience of the care management process. We then implemented a hybrid analytical approach that combines computer-assisted text analyses with deductive analyses as an alternative to conventional qualitative thematic analysis to identify word combination frequencies, content categories, and broad themes characterizing unmet needs in the care management process. RESULTS: Using this hybrid computer-assisted analytical approach, we were able to identify several key areas that are of interest to clinicians in the context of major depressive disorder and would be appropriate targets for digital tool development. CONCLUSIONS: A hybrid approach to qualitative research combining computer-assisted techniques with deductive techniques provides a time-efficient approach to identifying unmet needs, targets, and relevant themes to inform digital tool development. This can increase the likelihood that useful and practical tools are built and implemented to ultimately improve health outcomes for patients.

Feasibility, accuracy, and usability analysis of MapAML, a first-in-class app for integrated diagnosis in acute myeloid leukemia.

Performing a comprehensive diagnosis of acute myeloid leukemia (AML) is complex and involves the integration of clinical information, bone marrow morphology, immunophenotyping, cytogenetic, and molecular analysis, which can be challenging to the general hematologist. The aim of this study was to evaluate the usability and accuracy of MapAML, a smartphone app for integrated diagnosis in AML, created to aid the hematologist in its clinical practice. App performance was evaluated in dedicated sessions, in which 21 hematologists or fellows in hematology performed an integrated diagnosis of deidentified real-world clinical AML cases, first without and posteriorly with MapAML use. Diagnosis accuracy increased after MapAML utilization, with the average score going from 7.08 without app to 8.88 with app use (on a scale from 0 to 10), representing a significant accuracy improvement (p = .002). Usability evaluation was very favorable, with 81% of users considering the app very or extremely simple to use. There was also a significant increase in confidence to perform a complete and accurate diagnosis in AML after app use, with 61.9% of the participants willing to use the app in their clinical practice. In this study, MapAML increased accuracy with excellent usability for integrated diagnosis in AML.

Design of mobile and website health application devices for drug tolerability in hereditary fructose intolerance.

BACKGROUND: Hereditary fructose intolerance (HFI) is a rare metabolic disease caused by aldolase B deficiency. The aim of our study was to analyse excipient tolerability in patients with HFI and other related diseases and to design mobile and website health applications to facilitate the search for drugs according to their tolerance. RESULTS: A total of 555 excipients listed in the Spanish Medicines Agency database (July 2023) were classified as suitable for HFI patients, suitable with considerations ((glucose and glucose syrup, intravenous sucrose, oral mannitol, polydextrose, gums and carrageenans, ethanol, sulfite caramel and vanilla), not recommended (intravenous mannitol) and contraindicated (fructose, oral sucrose, invert sugar, sorbitol, maltitol, lactitol, isomaltitol, fruit syrups, honey, sucrose esters and sorbitol esters). Glucose and glucose syrup were classified as suitable with considerations due to its possible fructose content and their potential endogenous fructose production. For other related intolerances, wheat starch was contraindicated and oatmeal was not recommended in celiac disease; oral lactose and lactose-based coprocessed excipient (Cellactose®) were not recommended in lactose intolerance; and glucose, invert sugar and oral sucrose were not recommended in diabetes mellitus. The applications were named IntoMed®. Results are listed in order of tolerability (suitable drugs appear first and contraindicated drugs at the end), and they are accompanied by a note detailing their classified excipients. If a drug contains excipients within different categories, the overall classification will be the most restrictive. The apps are also able to classify substances with the same criteria if they act as active ingredients. The tools exhibited good usability (82.07 ± 13.46 points on the System Usability Scale [range: 0-100]) on a sample of HFI patients, their families and health care professionals. CONCLUSIONS: IntoMed® is a tool for finding information about the tolerability of drugs according to excipients for patients with HFI and other related intolerances, with good usability. It is a fast and reliable system that covers the current excipient legislation and expands on it with other specific information: HFI patients should be alert for excipients such as mannitol (especially in intravenous drugs), fruit syrups, honey, sulfite caramel or vanilla. Glucose might contain or produce fructose, and special precaution is needed because of potential errors in their composition.

Automatic dental age calculation from panoramic radiographs using deep learning: a two-stage approach with object detection and image classification.

BACKGROUND: Dental age is crucial for treatment planning in pediatric and orthodontic dentistry. Dental age calculation methods can be categorized into morphological, biochemical, and radiological methods. Radiological methods are commonly used because they are non-invasive and reproducible. When radiographs are available, dental age can be calculated by evaluating the developmental stage of permanent teeth and converting it into an estimated age using a table, or by measuring the length between some landmarks such as the tooth, root, or pulp, and substituting them into regression formulas. However, these methods heavily depend on manual time-consuming processes. In this study, we proposed a novel and completely automatic dental age calculation method using panoramic radiographs and deep learning techniques. METHODS: Overall, 8,023 panoramic radiographs were used as training data for Scaled-YOLOv4 to detect dental germs and mean average precision were evaluated. In total, 18,485 single-root and 16,313 multi-root dental germ images were used as training data for EfficientNetV2 M to classify the developmental stages of detected dental germs and Top-3 accuracy was evaluated since the adjacent stages of the dental germ looks similar and the many variations of the morphological structure can be observed between developmental stages. Scaled-YOLOv4 and EfficientNetV2 M were trained using cross-validation. We evaluated a single selection, a weighted average, and an expected value to convert the probability of developmental stage classification to dental age. One hundred and fifty-seven panoramic radiographs were used to compare automatic and manual human experts' dental age calculations. RESULTS: Dental germ detection was achieved with a mean average precision of 98.26% and dental germ classifiers for single and multi-root were achieved with a Top-3 accuracy of 98.46% and 98.36%, respectively. The mean absolute errors between the automatic and manual dental age calculations using single selection, weighted average, and expected value were 0.274, 0.261, and 0.396, respectively. The weighted average was better than the other methods and was accurate by less than one developmental stage error. CONCLUSION: Our study demonstrates the feasibility of automatic dental age calculation using panoramic radiographs and a two-stage deep learning approach with a clinically acceptable level of accuracy.

Application of neural networks for the detection of oral cancer: A systematic review.

One potential application of neural networks (NNs) is the early-stage detection of oral cancer. This systematic review aimed to determine the level of evidence on the sensitivity and specificity of NNs for the detection of oral cancer, following the Preferred Reporting Items for Systematic Reviews and MetaAnalyses (PRISMA) and Cochrane guidelines. Literature sources included PubMed, ClinicalTrials, Scopus, Google Scholar, and Web of Science. In addition, the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool was used to assess the risk of bias and the quality of the studies. Only 9 studies fully met the eligibility criteria. In most studies, NNs showed accuracy greater than 85%, though 100% of the studies presented a high risk of bias, and 33% showed high applicability concerns. Nonetheless, the included studies demonstrated that NNs were useful in the detection of oral cancer. However, studies of higher quality, with an adequate methodology, a low risk of bias and no applicability concerns are required so that more robust conclusions could be reached.

Best practices for multimodal clinical data management and integration: An atopic dermatitis research case.

BACKGROUND: In clinical research on multifactorial diseases such as atopic dermatitis, data-driven medical research has become more widely used as means to clarify diverse pathological conditions and to realize precision medicine. However, modern clinical data, characterized as large-scale, multimodal, and multi-center, causes difficulties in data integration and management, which limits productivity in clinical data science. METHODS: We designed a generic data management flow to collect, cleanse, and integrate data to handle different types of data generated at multiple institutions by 10 types of clinical studies. We developed MeDIA (Medical Data Integration Assistant), a software to browse the data in an integrated manner and extract subsets for analysis. RESULTS: MeDIA integrates and visualizes data and information on research participants obtained from multiple studies. It then provides a sophisticated interface that supports data management and helps data scientists retrieve the data sets they need. Furthermore, the system promotes the use of unified terms such as identifiers or sampling dates to reduce the cost of pre-processing by data analysts. We also propose best practices in clinical data management flow, which we learned from the development and implementation of MeDIA. CONCLUSIONS: The MeDIA system solves the problem of multimodal clinical data integration, from complex text data such as medical records to big data such as omics data from a large number of patients. The system and the proposed best practices can be applied not only to allergic diseases but also to other diseases to promote data-driven medical research.

[Functional requirements of patient data management systems in intensive care medicine]. / Funktionelle Anforderungen an Patientendatenmanagementsysteme in der Intensivmedizin.

BACKGROUND: As part of the German government's digitization initiative, the paper-based documentation that is still present in many intensive care units is to be replaced by digital patient data management systems (PDMS). In order to simplify the implementation of such systems, standards for basic functionalities that should be part of basic configurations of PDMS would be of great value. PURPOSE: This paper describes functional requirements for PDMS in several categories. METHODS: Criteria for standardized data documentation were defined by the authors and derived functional requirements were classified into two priority categories. RESULTS: Overall, general technical requirements, functionalities for intensive care patient care, and additional functionalities for PDMS were defined and prioritized. DISCUSSION: Using this paper as a starting point for a discussion about basic functionalities of PDMS, it is planned to develop and obtain consensus on definitive standards with representatives from medical societies, medical informatics and PDMS manufacture.

A tecnologia digital como mecanismo auxiliador no envelhecimento ativo no século XXI / Digital technology as an auxiliary mechanism for active aging in the 21st century

Objetivo: Analisar os efeitos benéficos do uso da tecnologia digital no envelhecimento ativo, considerando o contexto pós-pandemia Sars-Cov-2-covid-19. Além disso, pretende-se identificar as vantagens e desvantagens do uso dessas tecnologias pelas pessoas idosas no mundo contemporâneo. Método: estudo descritivo, tipo análise teórico reflexiva, desenvolvido a partir de duas questões norteadoras relacionados à temática, subsidiado por levantamento bibliográfico, considerando publicações pertinentes à temática, disponíveis nas bases de dados do Portal Regional da BVS, Portal de Periódicos Capes, SciELO e Pubmeb. por meio dos descritores controlados DECS /MeSH. Resultados: as TIC's estão desempenhando um papel crucial na vida das pessoas idosas, melhorando a comunicação, promovendo a saúde, facilitando o aprendizado e proporcionando um acesso mais fácil à informação e aos cuidados médicos. Conclusão: a tecnologia digital é um mecanismo auxiliador no envelhecimento ativo; quando bem implementado pode trazer mais vantagens do que desvantagens.(AU)

Objective: To analyze the beneficial effects of digital technology usage in active aging, considering the post-Sars-Cov-2-covid-19 pandemic context. Additionally, the aim is to identify the advantages and disadvantages of older individuals using these technologies in the contemporary world. Method: A descriptive study, specifically a theoretical reflective analysis, developed based on two guiding questions related to the theme. The study was supported by a literature review, considering publications relevant to the topic available in the databases of the Regional Portal of BVS, Capes Periodicals Portal, SciELO, and Pubmeb, using controlled descriptors such as DECS/MeSH. Results: Information and communication technologies (ICTs) are playing a crucial role in the lives of older individuals, enhancing communication, promoting health, facilitating learning, and providing easier access to information and medical care. Conclusion: Digital technology serves as an auxiliary mechanism in active aging; when well implemented, it can bring more advantages than disadvantages.(AU)

Objetivo: Analizar los efectos beneficiosos del uso de la tecnología digital en el envejecimiento activo, considerando el contexto post-pandemia de Sars-Cov-2-covid-19. Además, se pretende identificar las ventajas y desventajas del uso de estas tecnologías por parte de las personas mayores em mundo contemporáneo. Método: Estudio descriptivo, tipo análisis teórico reflexivo, desarrollado a partir de dos preguntas orientadoras relacionadas con el tema, respaldado por revisión bibliográfica que considera publicaciones pertinentes disponibles en las bases de datos del Portal Regional de BVS, Portal de Periódicos Capes, SciELO y Pubmeb, mediante descriptores controlados DECS/MeSH. Resultados: tecnologías de la información y la comunicación (TIC) desempeñan un papel crucial en la vida de las personas mayores, mejorando la comunicación, promoviendo la salud, facilitando aprendizaje y proporcionando acceso fácil a información y atención médica. Conclusión: La tecnología digital es un mecanismo auxiliar en el envejecimiento activo cuando se implementa adecuadamente puede aportar más ventajas que desventajas.(AU)

Stability monitoring of patients with myasthenia gravis using a mobile-based application.

Background: Failure in early diagnosis of myasthenia gravis (MG) and the risks of taking certain medications and undergoing surgery and anesthesia can lead to severe respiratory disorders and death. However, there are therapeutic measures that significantly control the disease and improve individual's functionality. Methods: First, an expert panel was formed, and a needs assessment questionnaire was prepared for the information elements and the capabilities required for the application and provided to neurologists with a subspecialty fellowship in neuromuscular diseases. Then, based on the analyzed results, the application was designed and created in 2 versions (physician and patient), and in 2 languages (Persian and English). Eventually, a questionnaire for user interaction and satisfaction was provided to 5 relevant physicians to evaluate the application. Results: The results showed that neurologists considered all items of the needs assessment questionnaire to be 100% essential. The capabilities of the application included registering the medication name and dose, recording symptoms and complaints by the patient, completing standard questionnaires, online chat, medication reminder, sending alerts to the doctor when the patient is unwell, and providing a variety of reports. The usability evaluation showed that neurologists evaluated the application at a good level with the average score of 8.23 ± 0.47 (out of 9 points). Conclusion: In the long run, using this technology can reduce costs, improve patients' quality of life (QOL) and health care, change health behaviors, and ultimately, improve individual's health.

Enhancing Primary Care for Nursing Home Patients with an Artificial Intelligence-Aided Rational Drug Use Web Assistant.

Polypharmacy can result in drug-drug interactions, severe side-effects, drug-disease interactions, inappropriate medication use in the elderly, and escalating costs. This study aims to evaluate nursing home residents' medication regimens using a rational drug use web assistant developed by researchers to mitigate unnecessary medication usage. This analytical, cross-sectional study included data from nursing home residents recently recorded in a training family health center. Sociodemographic information, medical conditions, and prescribed medications of all patients in the nursing home (n = 99) were documented. Medications were assessed using an artificial intelligence-aided rational drug use web assistant. Instances of inappropriate drug use and calculations of contraindicated drug costs were also recorded. The study revealed that 88.9% (n = 88) of patients experienced polypharmacy, with a mean value of 6.96 ± 2.94 drugs per patient. Potential risky drug-drug interactions were present in 89.9% (n = 89) of patients, contraindicated drug-drug interactions in 20.2% (n = 20), and potentially inappropriate drug use in 86.9% (n = 86). Plans to discontinue 83 medications were estimated to reduce total direct medication costs by 9.1% per month. After the assessment with the rational drug use web assistant, the number of drugs that patients needed to use and polypharmacy decreased significantly. This study concludes that the rational drug use web assistant application, which is more cost-effective than the traditional manual method, assisted by artificial intelligence, and integrated into healthcare services, may offer substantial benefits to family physicians and their geriatric patients.

ChatGPT's performance in dentistry and allergy-immunology assessments: a comparative study.

Large language models (LLMs) such as ChatGPT have potential applications in healthcare, including dentistry. Priming, the practice of providing LLMs with initial, relevant information, is an approach to improve their output quality. This study aimed to evaluate the performance of ChatGPT 3 and ChatGPT 4 on self-assessment questions for dentistry, through the Swiss Federal Licensing Examination in Dental Medicine (SFLEDM), and allergy and clinical immunology, through the European Examination in Allergy and Clinical Immunology (EEAACI). The second objective was to assess the impact of priming on ChatGPT's performance. The SFLEDM and EEAACI multiple-choice questions from the University of Bern's Institute for Medical Education platform were administered to both ChatGPT versions, with and without priming. Performance was analyzed based on correct responses. The statistical analysis included Wilcoxon rank sum tests (α=0.05). The average accuracy rates in the SFLEDM and EEAACI assessments were 63.3% and 79.3%, respectively. Both ChatGPT versions performed better on EEAACI than SFLEDM, with ChatGPT 4 outperforming ChatGPT 3 across all tests. ChatGPT 3's performance exhibited a significant improvement with priming for both EEAACI (p=0.017) and SFLEDM (p=0.024) assessments. For ChatGPT 4, the priming effect was significant only in the SFLEDM assessment (p=0.038). The performance disparity between SFLEDM and EEAACI assessments underscores ChatGPT's varying proficiency across different medical domains, likely tied to the nature and amount of training data available in each field. Priming can be a tool for enhancing output, especially in earlier LLMs. Advancements from ChatGPT 3 to 4 highlight the rapid developments in LLM technology. Yet, their use in critical fields such as healthcare must remain cautious owing to LLMs' inherent limitations and risks.

Verifying measurements on Siemens Atellica® instruments using clinically acceptable analytical performance specifications.

Measurements on clinical chemistry analysers must be verified to demonstrate applicability to their intended clinical use. We verified the performance of measurements on the Siemens Atellica® Solution chemistry analysers against the clinically acceptable analytical performance specifications, CAAPS, including the component of intra-individual biological variation, CVI. The relative standard uncertainty of measurement, i.e. analytical variation, CVA, was estimated for six example measurands, haemoglobin A1c in whole blood (B-HbA1c), albumin in urine (U-Alb), and the following measurands in plasma: sodium (P-Na), pancreatic amylase (P-AmylP), low-density lipoprotein cholesterol (P-LDL-C), and creatinine (P-Crea). Experimental CVA was calculated from single-instrument imprecision using control samples, variation between measurements on parallel instruments, and estimation of bias with pooled patient specimens. Each obtained CVA was compared with previously developed CAAPS. The calculated CVA was 1.4% for B-HbA1c (CAAPS 1.9% for single diagnostic testing, CAAPS 2.0% for monitoring after duplicate tests; IFCC units), 10.9% for U-Alb (CAAPS 44.9%), 1.2% for P-Na (CAAPS 0.6%, after triplicate testing 1.5%), 8.2% for P-AmylP (CAAPS 22.9%). The CVA was 4.9% for P-LDL-C (CAAPS for cardiovascular risk stratification 4.9% after four replicates), and 4.2% for P-Crea (CAAPS 8.0%). Three of the six measurands fulfilled the estimated clinical need. Results from P-Na measurements indicate a general need for improving the P-Na assays for emergency patients. It is necessary to consider CVI when creating diagnostic targets for laboratory tests, as emphasised by the CAAPS estimates of B-HbA1c and P-LDL-C.

Exploring patient medication adherence and data mining methods in clinical big data: A contemporary review.

BACKGROUND: Increasingly, patient medication adherence data are being consolidated from claims databases and electronic health records (EHRs). Such databases offer an indirect avenue to gauge medication adherence in our data-rich healthcare milieu. The surge in data accessibility, coupled with the pressing need for its conversion to actionable insights, has spotlighted data mining, with machine learning (ML) emerging as a pivotal technique. Nonadherence poses heightened health risks and escalates medical costs. This paper elucidates the synergistic interaction between medical database mining for medication adherence and the role of ML in fostering knowledge discovery. METHODS: We conducted a comprehensive review of EHR applications in the realm of medication adherence, leveraging ML techniques. We expounded on the evolution and structure of medical databases pertinent to medication adherence and harnessed both supervised and unsupervised ML paradigms to delve into adherence and its ramifications. RESULTS: Our study underscores the applications of medical databases and ML, encompassing both supervised and unsupervised learning, for medication adherence in clinical big data. Databases like SEER and NHANES, often underutilized due to their intricacies, have gained prominence. Employing ML to excavate patient medication logs from these databases facilitates adherence analysis. Such findings are pivotal for clinical decision-making, risk stratification, and scholarly pursuits, aiming to elevate healthcare quality. CONCLUSION: Advanced data mining in the era of big data has revolutionized medication adherence research, thereby enhancing patient care. Emphasizing bespoke interventions and research could herald transformative shifts in therapeutic modalities.

Computer programs used in the field of hospital pharmacy for the management of dangerous drugs: systematic review of literature.

Background: This review wants to highlight the importance of computer programs used to control the steps in the management of dangerous drugs. It must be taken into account that there are phases in the process of handling dangerous medicines in pharmacy services that pose a risk to the healthcare personnel who handle them. Objective: To review the scientific literature to determine what computer programs have been used in the field of hospital pharmacy for the management of dangerous drugs (HDs). Methods: The following electronic databases were searched from inception to July 30, 2021: MEDLINE (via PubMed), Embase, Cochrane Library, Scopus, Web of Science, Latin American and Caribbean Literature in Health Sciences (LILACS) and Medicine in Spanish (MEDES). The following terms were used in the search strategy: "Antineoplastic Agents," "Cytostatic Agents," "Hazardous Substances," "Medical Informatics Applications," "Mobile Applications," "Software," "Software Design," and "Pharmacy Service, Hospital." Results: A total of 104 studies were retrieved form the databases, and 18 additional studies were obtained by manually searching the reference lists of the included studies and by consulting experts. Once the inclusion and exclusion criteria were applied, 26 studies were ultimately included in this review. Most of the applications described in the included studies were used for the management of antineoplastic drugs. The most commonly controlled stage was electronic prescription; 18 studies and 7 interventions carried out in the preparation stage focused on evaluating the accuracy of chemotherapy preparations. Conclusion: Antineoplastic electronic prescription software was the most widely implemented software at the hospital level. No software was found to control the entire HD process. Only one of the selected studies measured safety events in workers who handle HDs. Moreover, health personnel were found to be satisfied with the implementation of this type of technology for daily work with these medications. All studies reviewed herein considered patient safety as their final objective. However, none of the studies evaluated the risk of HD exposure among workers.

Telehealth interventions during COVID-19 pandemic: a scoping review of applications, challenges, privacy and security issues.

BACKGROUND: The COVID-19, caused by the SARS-CoV-2 virus, proliferated worldwide, leading to a pandemic. Many governmental and non-governmental organisations and research institutes are contributing to the COVID-19 fight to control the pandemic. MOTIVATION: Numerous telehealth applications have been proposed and adopted during the pandemic to combat the spread of the disease. To this end, powerful tools such as artificial intelligence (AI)/robotic technologies, tracking, monitoring, consultation apps and other telehealth interventions have been extensively used. However, there are several issues and challenges that are currently facing this technology. OBJECTIVE: The purpose of this scoping review is to analyse the primary goal of these techniques; document their contribution to tackling COVID-19; identify and categorise their main challenges and future direction in fighting against the COVID-19 or future pandemic outbreaks. METHODS: Four digital libraries (ACM, IEEE, Scopus and Google Scholar) were searched to identify relevant sources. Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) was used as a guideline procedure to develop a comprehensive scoping review. General telehealth features were extracted from the studies reviewed and analysed in the context of the intervention type, technology used, contributions, challenges, issues and limitations. RESULTS: A collection of 27 studies were analysed. The reported telehealth interventions were classified into two main categories: AI-based and non-AI-based interventions; their main contributions to tackling COVID-19 are in the aspects of disease detection and diagnosis, pathogenesis and virology, vaccine and drug development, transmission and epidemic predictions, online patient consultation, tracing, and observation; 28 telehealth intervention challenges/issues have been reported and categorised into technical (14), non-technical (10), and privacy, and policy issues (4). The most critical technical challenges are: network issues, system reliability issues, performance, accuracy and compatibility issues. Moreover, the most critical non-technical issues are: the skills required, hardware/software cost, inability to entirely replace physical treatment and people's uncertainty about using the technology. Stringent laws/regulations, ethical issues are some of the policy and privacy issues affecting the development of the telehealth interventions reported in the literature. CONCLUSION: This study provides medical and scientific scholars with a comprehensive overview of telehealth technologies' current and future applications in the fight against COVID-19 to motivate researchers to continue to maximise the benefits of these techniques in the fight against pandemics. Lastly, we recommend that the identified challenges, privacy, and security issues and solutions be considered when designing and developing future telehealth applications.

Social Determinants of Health Documentation in Structured and Unstructured Clinical Data of Patients With Diabetes: Comparative Analysis.

Background: Electronic health records (EHRs) have yet to fully capture social determinants of health (SDOH) due to challenges such as nonexistent or inconsistent data capture tools across clinics, lack of time, and the burden of extra steps for the clinician. However, patient clinical notes (unstructured data) may be a better source of patient-related SDOH information. Objective: It is unclear how accurately EHR data reflect patients' lived experience of SDOH. The manual process of retrieving SDOH information from clinical notes is time-consuming and not feasible. We leveraged two high-throughput tools to identify SDOH mappings to structured and unstructured patient data: PatientExploreR and Electronic Medical Record Search Engine (EMERSE). Methods: We included adult patients (≥18 years of age) receiving primary care for their diabetes at the University of California, San Francisco (UCSF), from January 1, 2018, to December 31, 2019. We used expert raters to develop a corpus using SDOH in the compendium as a knowledge base as targets for the natural language processing (NLP) text string mapping to find string stems, roots, and syntactic similarities in the clinical notes of patients with diabetes. We applied advanced built-in EMERSE NLP query parsers implemented with JavaCC. Results: We included 4283 adult patients receiving primary care for diabetes at UCSF. Our study revealed that SDOH may be more significant in the lives of patients with diabetes than is evident from structured data recorded on EHRs. With the application of EMERSE NLP rules, we uncovered additional information from patient clinical notes on problems related to social connectionsisolation, employment, financial insecurity, housing insecurity, food insecurity, education, and stress. Conclusions: We discovered more patient information related to SDOH in unstructured data than in structured data. The application of this technique and further investment in similar user-friendly tools and infrastructure to extract SDOH information from unstructured data may help to identify the range of social conditions that influence patients' disease experiences and inform clinical decision-making.

A lifecycle framework illustrates eight stages necessary for realizing the benefits of patient-centered clinical decision support.

The design, development, implementation, use, and evaluation of high-quality, patient-centered clinical decision support (PC CDS) is necessary if we are to achieve the quintuple aim in healthcare. We developed a PC CDS lifecycle framework to promote a common understanding and language for communication among researchers, patients, clinicians, and policymakers. The framework puts the patient, and/or their caregiver at the center and illustrates how they are involved in all the following stages: Computable Clinical Knowledge, Patient-specific Inference, Information Delivery, Clinical Decision, Patient Behaviors, Health Outcomes, Aggregate Data, and patient-centered outcomes research (PCOR) Evidence. Using this idealized framework reminds key stakeholders that developing, deploying, and evaluating PC-CDS is a complex, sociotechnical challenge that requires consideration of all 8 stages. In addition, we need to ensure that patients, their caregivers, and the clinicians caring for them are explicitly involved at each stage to help us achieve the quintuple aim.

New Artificial Intelligence ChatGPT Performs Poorly on the 2022 Self-assessment Study Program for Urology.

INTRODUCTION: Large language models have demonstrated impressive capabilities, but application to medicine remains unclear. We seek to evaluate the use of ChatGPT on the American Urological Association Self-assessment Study Program as an educational adjunct for urology trainees and practicing physicians. METHODS: One hundred fifty questions from the 2022 Self-assessment Study Program exam were screened, and those containing visual assets (n=15) were removed. The remaining items were encoded as open ended or multiple choice. ChatGPT's output was coded as correct, incorrect, or indeterminate; if indeterminate, responses were regenerated up to 2 times. Concordance, quality, and accuracy were ascertained by 3 independent researchers and reviewed by 2 physician adjudicators. A new session was started for each entry to avoid crossover learning. RESULTS: ChatGPT was correct on 36/135 (26.7%) open-ended and 38/135 (28.2%) multiple-choice questions. Indeterminate responses were generated in 40 (29.6%) and 4 (3.0%), respectively. Of the correct responses, 24/36 (66.7%) and 36/38 (94.7%) were on initial output, 8 (22.2%) and 1 (2.6%) on second output, and 4 (11.1%) and 1 (2.6%) on final output, respectively. Although regeneration decreased indeterminate responses, proportion of correct responses did not increase. For open-ended and multiple-choice questions, ChatGPT provided consistent justifications for incorrect answers and remained concordant between correct and incorrect answers. CONCLUSIONS: ChatGPT previously demonstrated promise on medical licensing exams; however, application to the 2022 Self-assessment Study Program was not demonstrated. Performance improved with multiple-choice over open-ended questions. More importantly were the persistent justifications for incorrect responses-left unchecked, utilization of ChatGPT in medicine may facilitate medical misinformation.